Random effects meta-analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation

A random effects meta-analysis combines the results of several independent studies to summarise the evidence about a particular measure of interest, such as a treatment effect. The approach allows for unexplained between-study heterogeneity in the true treatment effect by incorporating random study effects about the overall mean. The variance of the mean effect estimate is conventionally calculated by assuming that the between study variance is known; however, it has been demonstrated that this approach may be inappropriate, especially when there are few studies. Alternative methods that aim to account for this uncertainty, such as Hartung-Knapp, Sidik-Jonkman and Kenward-Roger, have been proposed and shown to improve upon the conventional approach in some situations. In this paper, we use a simulation study to examine the performance of several of these methods in terms of the coverage of the 95% confidence and prediction intervals derived from a random effects meta-analysis estimated using restricted maximum likelihood. We show that, in terms of the confidence intervals, the Hartung-Knapp correction performs well across a wide-range of scenarios and outperforms other methods when heterogeneity was large and/or study sizes were similar. However, the coverage of the Hartung-Knapp method is slightly too low when the heterogeneity is low (I2 30%) and study sizes are similar. In other situations, especially when heterogeneity is small and the study sizes are quite varied, the coverage is far too low and could not be consistently improved by either increasing the number of studies, altering the degrees of freedom or using variance inflation methods. Therefore, researchers should be cautious in deriving 95% prediction intervals following a frequentist random-effects meta-analysis until a more reliable solution is identifie

Outcome assessment by central adjudicators in randomised stroke trials: Simulation of differential and non-differential misclassification

INTRODUCTION: Adjudication of the primary outcome in randomised trials is thought to control misclassification. We investigated the amount of misclassification needed before adjudication changed the primary trial results. Patients (or materials) and methods: We included data from five randomised stroke trials. Differential misclassification was introduced for each primary outcome until the estimated treatment effect was altered. This was simulated 1000 times. We calculated the between-simulation mean proportion of participants that needed to be differentially misclassified to alter the treatment effect. In addition, we simulated hypothetical trials with a binary outcome and varying sample size (1000–10,000), overall event rate (10%–50%) and treatment effect (0.67–0.90). We introduced non-differential misclassification until the treatment effect was non-significant at 5% level. RESULTS: For the five trials, the range of unweighted kappa values were reduced from 0.89–0.97 to 0.65–0.85 before the treatment effect was altered. This corresponded to 2.1%–6% of participants misclassified differentially for trials with a binary outcome. For the hypothetical trials, those with a larger sample size, stronger treatment effect and overall event rate closer to 50% needed a higher proportion of events non-differentially misclassified before the treatment effect became non-significant. DISCUSSION: We found that only a small amount of differential misclassification was required before adjudication altered the primary trial results, whereas a considerable proportion of participants needed to be misclassified non-differentially before adjudication changed trial conclusions. Given that differential misclassification should not occur in trials with sufficient blinding, these results suggest that central adjudication is of most use in studies with unblinded outcome assessment. CONCLUSION: For trials without adequate blinding, central adjudication is vital to control for differential misclassification. However, for large blinded trials, adjudication is of less importance and may not be necessary

Prophylactic antibiotics in the prevention of infection after operative vaginal delivery (ANODE): a multicentre randomised controlled trial.

BACKGROUND: Risk factors for maternal infection are clearly recognised, including caesarean section and operative vaginal birth. Antibiotic prophylaxis at caesarean section is widely recommended because there is clear systematic review evidence that it reduces incidence of maternal infection. Current WHO guidelines do not recommend routine antibiotic prophylaxis for women undergoing operative vaginal birth because of insufficient evidence of effectiveness. We aimed to investigate whether antibiotic prophylaxis prevented maternal infection after operative vaginal birth. METHODS: In a blinded, randomised controlled trial done at 27 UK obstetric units, women (aged ≥16 years) were allocated to receive a single dose of intravenous amoxicillin and clavulanic acid or placebo (saline) following operative vaginal birth at 36 weeks gestation or later. The primary outcome was confirmed or suspected maternal infection within 6 weeks of delivery defined by a new prescription of antibiotics for specific indications, confirmed systemic infection on culture, or endometritis. We did an intention-to-treat analysis. This trial is registered with ISRCTN, number 11166984, and is closed to accrual. FINDINGS: Between March 13, 2016, and June 13, 2018, 3427 women were randomly assigned to treatment: 1719 to amoxicillin and clavulanic acid, and 1708 to placebo. Seven women withdrew, leaving 1715 in the amoxicillin and clavulanic acid group and 1705 in the placebo groups. Primary outcome data were missing for 195 (6%) women. Significantly fewer women allocated to amoxicillin and clavulanic acid had a confirmed or suspected infection (180 [11%] of 1619) than women allocated to placebo (306 [19%] of 1606; risk ratio 0·58, 95% CI 0·49-0·69; p<0·0001). One woman in the placebo group reported a skin rash and two women in the amoxicillin and clavulanic acid reported other allergic reactions, one of which was reported as a serious adverse event. Two other serious adverse events were reported, neither was considered causally related to the treatment. INTERPRETATION: This trial shows benefit of a single dose of prophylactic antibiotic after operative vaginal birth and guidance from WHO and other national organisations should be changed to reflect this. FUNDING: NIHR Health Technology Assessment programme

Enteral lactoferrin to prevent infection for very preterm infants: the ELFIN RCT.

BACKGROUND: Infections acquired in hospital are an important cause of morbidity and mortality in very preterm infants. Several small trials have suggested that supplementing the enteral diet of very preterm infants with lactoferrin, an antimicrobial protein processed from cow's milk, prevents infections and associated complications. OBJECTIVE: To determine whether or not enteral supplementation with bovine lactoferrin (The Tatua Cooperative Dairy Company Ltd, Morrinsville, New Zealand) reduces the risk of late-onset infection (acquired > 72 hours after birth) and other morbidity and mortality in very preterm infants. DESIGN: Randomised, placebo-controlled, parallel-group trial. Randomisation was via a web-based portal and used an algorithm that minimised for recruitment site, weeks of gestation, sex and single versus multiple births. SETTING: UK neonatal units between May 2014 and September 2017. PARTICIPANTS: Infants born at < 32 weeks' gestation and aged < 72 hours at trial enrolment. INTERVENTIONS: Eligible infants were allocated individually (1 : 1 ratio) to receive enteral bovine lactoferrin (150 mg/kg/day; maximum 300 mg/day) or sucrose (British Sugar, Peterborough, UK) placebo (same dose) once daily from trial entry until a postmenstrual age of 34 weeks. Parents, caregivers and outcome assessors were unaware of group assignment. OUTCOMES: Primary outcome - microbiologically confirmed or clinically suspected late-onset infection. Secondary outcomes - microbiologically confirmed infection; all-cause mortality; severe necrotising enterocolitis (NEC); retinopathy of prematurity (ROP); bronchopulmonary dysplasia (BPD); a composite of infection, NEC, ROP, BPD and mortality; days of receipt of antimicrobials until 34 weeks' postmenstrual age; length of stay in hospital; and length of stay in intensive care, high-dependency and special-care settings. RESULTS: Of 2203 enrolled infants, primary outcome data were available for 2182 infants (99%). In the intervention group, 316 out of 1093 (28.9%) infants acquired a late-onset infection versus 334 out of 1089 (30.7%) infants in the control group [adjusted risk ratio (RR) 0.95, 95% confidence interval (CI) 0.86 to 1.04]. There were no significant differences in any secondary outcomes: microbiologically confirmed infection (RR 1.05, 99% CI 0.87 to 1.26), mortality (RR 1.05, 99% CI 0.66 to 1.68), NEC (RR 1.13, 99% CI 0.68 to 1.89), ROP (RR 0.89, 99% CI 0.62 to 1.28), BPD (RR 1.01, 99% CI 0.90 to 1.13), or a composite of infection, NEC, ROP, BPD and mortality (RR 1.01, 99% CI 0.94 to 1.08). There were no differences in the number of days of receipt of antimicrobials, length of stay in hospital, or length of stay in intensive care, high-dependency or special-care settings. There were 16 reports of serious adverse events for infants in the lactoferrin group and 10 for infants in the sucrose group. CONCLUSIONS: Enteral supplementation with bovine lactoferrin does not reduce the incidence of infection, mortality or other morbidity in very preterm infants. FUTURE WORK: Increase the precision of the estimates of effect on rarer secondary outcomes by combining the data in a meta-analysis with data from other trials. A mechanistic study is being conducted in a subgroup of trial participants to explore whether or not lactoferrin supplementation affects the intestinal microbiome and metabolite profile of very preterm infants. TRIAL REGISTRATION: Current Controlled Trials ISRCTN88261002. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 74. See the NIHR Journals Library website for further project information. This trial was also sponsored by the University of Oxford, Oxford, UK. The funder provided advice and support and monitored study progress but did not have a role in study design or data collection, analysis and interpretation

Intravenous co-amoxiclav to prevent infection after operative vaginal delivery: the ANODE RCT.

BACKGROUND: Sepsis is a leading cause of direct and indirect maternal death in both the UK and globally. All forms of operative delivery are associated with an increased risk of sepsis, and the National Institute for Health and Care Excellence's guidance recommends the use of prophylactic antibiotics at all caesarean deliveries, based on substantial randomised controlled trial evidence of clinical effectiveness. A Cochrane review, updated in 2017 (Liabsuetrakul T, Choobun T, Peeyananjarassri K, Islam QM. Antibiotic prophylaxis for operative vaginal delivery. Cochrane Database Syst Rev 2017;8:CD004455), identified only one small previous trial of prophylactic antibiotics following operative vaginal birth (forceps or ventouse/vacuum extraction) and, given the small study size and extreme result, suggested that further robust evidence is needed. OBJECTIVES: To investigate whether or not a single dose of prophylactic antibiotic following operative vaginal birth is clinically effective for preventing confirmed or presumed maternal infection, and to investigate the associated impact on health-care costs. DESIGN: A multicentre, randomised, blinded, placebo-controlled trial. SETTING: Twenty-seven maternity units in the UK. PARTICIPANTS: Women who had an operative vaginal birth at ≥ 36 weeks' gestation, who were not known to be allergic to penicillin or constituents of co-amoxiclav and who had no indication for ongoing antibiotics. INTERVENTIONS: A single dose of intravenous co-amoxiclav (1 g of amoxicillin/200 mg of clavulanic acid) or placebo (sterile saline) allocated through sealed, sequentially numbered, indistinguishable packs. MAIN OUTCOME MEASURES: Primary outcome - confirmed or suspected infection within 6 weeks of giving birth. Secondary outcomes - severe sepsis, perineal wound infection, perineal pain, use of pain relief, hospital bed stay, hospital/general practitioner visits, need for additional perineal care, dyspareunia, ability to sit comfortably to feed the baby, maternal general health, breastfeeding, wound breakdown, occurrence of anaphylaxis and health-care costs. RESULTS: Between March 2016 and June 2018, 3427 women were randomised: 1719 to the antibiotic arm and 1708 to the placebo arm. Seven women withdrew, leaving 1715 women in the antibiotic arm and 1705 in the placebo arm for analysis. Primary outcome data were available for 3225 out of 3420 women (94.3%). Women randomised to the antibiotic arm were significantly less likely to have confirmed or suspected infection within 6 weeks of giving birth (180/1619, 11%) than women randomised to the placebo arm (306/1606, 19%) (relative risk 0.58, 95% confidence interval 0.49 to 0.69). Three serious adverse events were reported: one in the placebo arm and two in the antibiotic arm (one was thought to be causally related to the intervention). LIMITATIONS: The follow-up rate achieved for most secondary outcomes was 76%. CONCLUSIONS: This trial has shown clear evidence of benefit of a single intravenous dose of prophylactic co-amoxiclav after operative vaginal birth. These results may lead to reconsideration of official policy/guidance. Further analysis of the mechanism of action of this single dose of antibiotic is needed to investigate whether earlier, pre-delivery or repeated administration could be more effective. Until these analyses are completed, there is no indication for administration of more than a single dose of prophylactic antibiotic, or for pre-delivery administration. TRIAL REGISTRATION: Current Controlled Trials ISRCTN11166984. FUNDING: This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 23, No. 54. See the National Institute for Health Research Journals Library website for further project information

Enteral lactoferrin supplementation for very preterm infants: a randomised placebo-controlled trial

Background Infections acquired in hospital are an important cause of morbidity and mortality in very preterm infants. Several small trials have suggested that supplementing the enteral diet of very preterm infants with lactoferrin, an antimicrobial protein processed from cow's milk, prevents infections and associated complications. The aim of this large randomised controlled trial was to collect data to enhance the validity and applicability of the evidence from previous trials to inform practice. Methods In this randomised placebo-controlled trial, we recruited very preterm infants born before 32 weeks' gestation in 37 UK hospitals and younger than 72 h at randomisation. Exclusion criteria were presence of a severe congenital anomaly, anticipated enteral fasting for longer than 14 days, or no realistic prospect of survival. Eligible infants were randomly assigned (1:1) to receive either enteral bovine lactoferrin (150 mg/kg per day; maximum 300 mg/day; lactoferrin group) or sucrose (same dose; control group) once daily until 34 weeks' postmenstrual age. Web-based randomisation minimised for recruitment site, gestation (completed weeks), sex, and single versus multifetal pregnancy. Parents, caregivers, and outcome assessors were unaware of group assignment. The primary outcome was microbiologically confirmed or clinically suspected late-onset infection (occurring >72 h after birth), which was assessed in all participants for whom primary outcome data was available by calculating the relative risk ratio with 95% CI between the two groups. The trial is registered with the International Standard Randomised Controlled Trial Number 88261002. Findings We recruited 2203 participants between May 7, 2014, and Sept 28, 2017, of whom 1099 were assigned to the lactoferrin group and 1104 to the control group. Four infants had consent withdrawn or unconfirmed, leaving 1098 infants in the lactoferrin group and 1101 in the sucrose group. Primary outcome data for 2182 infants (1093 [99·5%] of 1098 in the lactoferrin group and 1089 [99·0] of 1101 in the control group) were available for inclusion in the modified intention-to-treat analyses. 316 (29%) of 1093 infants in the intervention group acquired a late-onset infection versus 334 (31%) of 1089 in the control group. The risk ratio adjusted for minimisation factors was 0·95 (95% CI 0·86–1·04; p=0·233). During the trial there were 16 serious adverse events for infants in the lactoferrin group and 10 for infants in the control group. Two events in the lactoferrin group (one case of blood in stool and one death after intestinal perforation) were assessed as being possibly related to the trial intervention. Interpretation Enteral supplementation with bovine lactoferrin does not reduce the risk of late-onset infection in very preterm infants. These data do not support its routine use to prevent late-onset infection and associated morbidity or mortality in very preterm infants. Funding UK National Institute for Health Research Health Technology Assessment programme (10/57/49)

Enteral lactoferrin supplementation for very preterm infants: a randomised placebo-controlled trial

Background Infections acquired in hospital are an important cause of morbidity and mortality in very preterm infants. Several small trials have suggested that supplementing the enteral diet of very preterm infants with lactoferrin, an antimicrobial protein processed from cow's milk, prevents infections and associated complications. The aim of this large randomised controlled trial was to collect data to enhance the validity and applicability of the evidence from previous trials to inform practice. Methods In this randomised placebo-controlled trial, we recruited very preterm infants born before 32 weeks' gestation in 37 UK hospitals and younger than 72 h at randomisation. Exclusion criteria were presence of a severe congenital anomaly, anticipated enteral fasting for longer than 14 days, or no realistic prospect of survival. Eligible infants were randomly assigned (1:1) to receive either enteral bovine lactoferrin (150 mg/kg per day; maximum 300 mg/day; lactoferrin group) or sucrose (same dose; control group) once daily until 34 weeks' postmenstrual age. Web-based randomisation minimised for recruitment site, gestation (completed weeks), sex, and single versus multifetal pregnancy. Parents, caregivers, and outcome assessors were unaware of group assignment. The primary outcome was microbiologically confirmed or clinically suspected late-onset infection (occurring >72 h after birth), which was assessed in all participants for whom primary outcome data was available by calculating the relative risk ratio with 95% CI between the two groups. The trial is registered with the International Standard Randomised Controlled Trial Number 88261002. Findings We recruited 2203 participants between May 7, 2014, and Sept 28, 2017, of whom 1099 were assigned to the lactoferrin group and 1104 to the control group. Four infants had consent withdrawn or unconfirmed, leaving 1098 infants in the lactoferrin group and 1101 in the sucrose group. Primary outcome data for 2182 infants (1093 [99·5%] of 1098 in the lactoferrin group and 1089 [99·0] of 1101 in the control group) were available for inclusion in the modified intention-to-treat analyses. 316 (29%) of 1093 infants in the intervention group acquired a late-onset infection versus 334 (31%) of 1089 in the control group. The risk ratio adjusted for minimisation factors was 0·95 (95% CI 0·86–1·04; p=0·233). During the trial there were 16 serious adverse events for infants in the lactoferrin group and 10 for infants in the control group. Two events in the lactoferrin group (one case of blood in stool and one death after intestinal perforation) were assessed as being possibly related to the trial intervention. Interpretation Enteral supplementation with bovine lactoferrin does not reduce the risk of late-onset infection in very preterm infants. These data do not support its routine use to prevent late-onset infection and associated morbidity or mortality in very preterm infants. Funding UK National Institute for Health Research Health Technology Assessment programme (10/57/49)

Interferon gamma release assays for Diagnostic Evaluation of Active tuberculosis (IDEA): test accuracy study and economic evaluation

Background Interferon gamma release assays (IGRAs) are blood tests recommended for the diagnosis of tuberculosis (TB) infection. There is currently uncertainty about the role and clinical utility of IGRAs in the diagnostic workup of suspected active TB in routine NHS clinical practice. Objectives To compare the diagnostic accuracy and cost-effectiveness of T-SPOT.TB® (Oxford Immunotec, Abingdon, UK) and QuantiFERON® TB GOLD In-Tube (Cellestis, Carnegie, VIC, Australia) for diagnosis of suspected active TB and to estimate the diagnostic accuracy of second-generation IGRAs. Design Prospective within-patient comparative diagnostic accuracy study. Setting Secondary care. Participants Adults (aged ≥ 16 years) presenting as inpatients or outpatients at 12 NHS hospital trusts in London, Slough, Oxford, Leicester and Birmingham with suspected active TB. Interventions The index tests [T-SPOT.TB and QuantiFERON GOLD In-Tube (QFT-GIT)] and new enzyme-linked immunospot assays utilising novel Mycobacterium tuberculosis antigens (Rv3615c, Rv2654, Rv3879c and Rv3873) were verified against a composite reference standard applied by a panel of clinical experts blinded to IGRA results. Main outcome measures Sensitivity, specificity, predictive values and likelihood ratios were calculated to determine diagnostic accuracy. A decision tree model was developed to calculate the incremental costs and incremental health utilities [quality-adjusted life-years (QALYs)] of changing from current practice to using an IGRA as an initial rule-out test. Results A total of 363 patients had active TB (culture-confirmed and highly probable TB cases), 439 had no active TB and 43 had an indeterminate final diagnosis. Comparing T-SPOT.TB and QFT-GIT, the sensitivities [95% confidence interval (CI)] were 82.3% (95% CI 77.7% to 85.9%) and 67.3% (95% CI 62.1% to 72.2%), respectively, whereas specificities were 82.6% (95% CI 78.6% to 86.1%) and 80.4% (95% CI 76.1% to 84.1%), respectively. T-SPOT.TB was more sensitive than QFT-GIT (relative sensitivity 1.22, 95% CI 1.14 to 1.31; p < 0.001), but the specificities were similar (relative specificity 1.02, 95% CI 0.97 to 1.08; p = 0.3). For both IGRAs the sensitivity was lower and the specificity was higher for human immunodeficiency virus (HIV)-positive than for HIV-negative patients. The most promising novel antigen was Rv3615c. The added value of Rv3615c to T-SPOT.TB was a 9% (95% CI 5% to 12%) relative increase in sensitivity at the expense of specificity, which had a relative decrease of 7% (95% CI 4% to 10%). The use of current IGRA tests for ruling out active TB is unlikely to be considered cost-effective if a QALY was valued at £20,000 or £30,000. For T-SPOT.TB, the probability of being cost-effective for a willingness to pay of £20,000/QALY was 26% and 21%, when patients with indeterminate test results were excluded or included, respectively. In comparison, the QFT-GIT probabilities were 8% and 6%. Although the use of IGRAs is cost saving, the health detriment is large owing to delay in diagnosing active TB, leading to prolonged illness. There was substantial between-patient variation in the tests used in the diagnostic pathway. Limitations The recruitment target for the HIV co-infected population was not achieved. Conclusions Although T-SPOT.TB was more sensitive than QFT-GIT for the diagnosis of active TB, the tests are insufficiently sensitive for ruling out active TB in routine clinical practice in the UK. Novel assays offer some promise. Future work The novel assays require evaluation in distinct clinical settings and in immunosuppressed patient groups. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and the NIHR Health Protection Research Unit in Respiratory Infections, Imperial College London, London, UK

BHPR research: qualitative1. Complex reasoning determines patients' perception of outcome following foot surgery in rheumatoid arhtritis

Background: Foot surgery is common in patients with RA but research into surgical outcomes is limited and conceptually flawed as current outcome measures lack face validity: to date no one has asked patients what is important to them. This study aimed to determine which factors are important to patients when evaluating the success of foot surgery in RA Methods: Semi structured interviews of RA patients who had undergone foot surgery were conducted and transcribed verbatim. Thematic analysis of interviews was conducted to explore issues that were important to patients. Results: 11 RA patients (9 ♂, mean age 59, dis dur = 22yrs, mean of 3 yrs post op) with mixed experiences of foot surgery were interviewed. Patients interpreted outcome in respect to a multitude of factors, frequently positive change in one aspect contrasted with negative opinions about another. Overall, four major themes emerged. Function: Functional ability & participation in valued activities were very important to patients. Walking ability was a key concern but patients interpreted levels of activity in light of other aspects of their disease, reflecting on change in functional ability more than overall level. Positive feelings of improved mobility were often moderated by negative self perception ("I mean, I still walk like a waddling duck”). Appearance: Appearance was important to almost all patients but perhaps the most complex theme of all. Physical appearance, foot shape, and footwear were closely interlinked, yet patients saw these as distinct separate concepts. Patients need to legitimize these feelings was clear and they frequently entered into a defensive repertoire ("it's not cosmetic surgery; it's something that's more important than that, you know?”). Clinician opinion: Surgeons' post operative evaluation of the procedure was very influential. The impact of this appraisal continued to affect patients' lasting impression irrespective of how the outcome compared to their initial goals ("when he'd done it ... he said that hasn't worked as good as he'd wanted to ... but the pain has gone”). Pain: Whilst pain was important to almost all patients, it appeared to be less important than the other themes. Pain was predominately raised when it influenced other themes, such as function; many still felt the need to legitimize their foot pain in order for health professionals to take it seriously ("in the end I went to my GP because it had happened a few times and I went to an orthopaedic surgeon who was quite dismissive of it, it was like what are you complaining about”). Conclusions: Patients interpret the outcome of foot surgery using a multitude of interrelated factors, particularly functional ability, appearance and surgeons' appraisal of the procedure. While pain was often noted, this appeared less important than other factors in the overall outcome of the surgery. Future research into foot surgery should incorporate the complexity of how patients determine their outcome Disclosure statement: All authors have declared no conflicts of interes